Repository navigation
Conversation
Add `verify.sh compare`, which checks each run against a new "Expected step status" table in every feature file. Make the recorded crash site the same in every worktree and run. Drive the map's missing entry points (every dosplot mode, plot_quiver and overlay_orbitals, the zone in BS2DPlotter, FermiHandler save_3d and the box widget, Elk, Abinit and QE non-collinear parsers, spin_asymmetry, download_from_hf and the procar.py CLI). Correct the claims that object-API energies are Fermi-shifted and re-pin the proven end states to c13166c.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR maintains the
verify-pyprocarskill at devc13166ce. Each feature file now has an expected status for every driver step, and the newverify.sh comparechecks runs against those statuses. The drivers now cover the public entry points that the map did not drive. The map no longer says that object-API energies are Fermi-shifted. The proven end states are re-pinned toc13166ce.Only files in
.claude/skills/verify-pyprocar/change. No product code changes.What changed
verify.sh compare <run-dir>...(scripts/compare.py). It finds the driver by the bytes ofevidence/driver.py. It then checkssummary.json,side_effects.txtand the exit code against the## Expected step statustable in the feature files. Each row isok, orknown-defectwith an error type and a crash site. It exits 1 when any run deviates.verify_stepsrecordedwherefrom the checkout directory name. In a worktree it gaveplotter/bs_plot.py:986, a matplotlib frame, or a path that held the run directory. Now it ispyprocar/<path>:<line>, elsedriver.py:<line>.bands.py:plot_overlaywith an array,plot_quivergivenebs.bands, legacyoverlay_orbitals.dos.py: the sevendosplotmodes that were not driven.bs2d.py:plot(show_brillouin_zone=True).fermi3d.py:FermiHandlersave_3dandplot_fermi_cross_section_box_widget.fermi2d.py:plain_bands.parsers.py: Elk, Abinit and QE non-collinear bands.utilities.py:spin_asymmetry,download_from_hfwith a str path, andscripts/procar.pycat,filter,bandgapandgenerate2dkmesh, run in-process.from_codekeeps raw energies for bands and DOS. The map said they were Fermi-shifted. Bands ylim[-31.97, 13.59]is the raw PROCAR range plus 10%. DOS starts at-31.0611, the first vasprun row.BS2DPlotter.plot(show_brillouin_zone=True)works. The map listed it as a crash.FermiHandlersave_3dexports meshes. The map said it only logs "not yet implemented".None, so itsarea_textcannot be read.dos-rewritenote, the deletednew_ebs_examples.pygotcha, the QE "until Projection layout follow-ups: non-collinear spin channels, axis order and orbital names per parser #293" sentence, the Small correctness and cleanup follow-ups from the #277-#279 round #285/Brillouin zone and cross-section zone are wrong for non-reduced reciprocal bases #286 attributions (the fixes were fix: resolve the #285 follow-ups (DOS labels, autobandsplot report, BS2D grid, overlay weights, Elk lmirep) #304 and fix(zone): reduce the reciprocal basis before building the Brillouin zone #295), and the bs2d color counts.features/README.mdnow lists the entry points that no feature file covers, with the reason for each.SKILL.mdnow runs all 11 drivers and ends withcompare.Verification
All runs are in
data/verify-runs/of the main checkout, driven from a worktree atc13166ceplus this change.SKILL.md, tagmap232618:*-final-{bands,utilities,dos}, evidence written at 23:28) gives0 of 3 run(s) deviate.comparefails when it should. I planted one violation in each of 7 scratch copies of thebandsrun. The violations were anebs.pklside effect, a failedokstep, a fixed defect, a moved crash site, an unmapped step, a wrong exit code and an edited driver. The output was7 of 8 run(s) deviate, and the unchanged copy matched.basedpyright --project pyrightconfig.json: 0 errors, 0 warnings.ruff_new_violations.py check|format origin/dev: 9 changed Python files, ok.pytest tests/test_verify_fixtures.py tests/test_fixture_copies.py: 173 passed, 2 skipped.obj_bs2d_bz.pngshows the hexagonal prism.legacy_box_widget_slice.pngshows the closed orbits around Γ.legacy_stack_species.pngshows the stacked DOS under the total.obj_plain.pngshows E_F near +5 eV.Product gaps found (not fixed here)
The map records these as
known-defectrows. Each one reproduces in theutilitiesordosrun.download_from_hf(relpath, output_path="."):TypeErroratpyprocar/utils/download_examples.py:143. The installedpyprocar-downloadCLI hits the same error for every relpath, printsDownload failedand exits 1.python scripts/procar.py bandgap PROCAR OUTCAR vasp 5.3017:ValueError: Invalid code: 5.3017. Its arguments go in the wrong order atscripts/procar.py:124.python scripts/procar.py generate2dkmesh -0.5 -0.5 0.5 0.5 0 5 4:TypeErroratpyprocar/scripts/scriptKmesh2D.py:42. Its arguments have notype=.dosplot(mode="stack")withoutitems:IndexErroratpyprocar/scripts/scriptDosplot.py:427.pyprocar.spin_asymmetryis still exported, but it always raisesNotImplementedError.Not verified
data/codes.zipby$H fetch data/codes. The source suggests that it extracts todata/codes/codes/. I did not download it.procar.py kpathdefaults-with_time_reversalto False, butkpath()defaults it to True. This is from source only.data/store has 2223 writable fixture paths, because the fixtures predate feat(verify-pyprocar): make fetched fixtures read-only and keep verify.sh inside data/ #303. I did not lock them. The main checkout is on an olderdev.